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Abstract 

We consider a spiked population model, proposed by Johnstone, whose population eigenvalues are all 
unit except for a few fixed eigenvalues. The question is to determine how the sample eigenvalues depend 
on the non-unit population ones when both sample size and population size become large. This paper 
completely determines the almost sure limits for a general class of samples. 

1 Introduction 

The sample covariance matrix is fundamental to multivariate statistics. When the population size is not 
large and for a sufficient number of samples, the sample covariance matrix is a good approximate of the 
population covariance matrix . However when the population size is large and comparable with the sample 
size, as is in many contemporary data, it is known that the sample covariance matrix is no longer a good 
approximation to the covariance matrix. The Marchenko-Pastur theorem states that with n = the 
sample size, p = the population size, as n = n{p) — > oo such that ^ ^ c, the eigenvalues s^^, j = 1, . . . ,p, 
of the sample covariance matrix of normalized i.i.d. Gaussian samples satisfy for any real x 

i#{sf : <x}^ F{x) (1.1) 
p J J 

almost surely where 

F'{x) = -^^y{b- x){x - a), a<x<b, (1.2) 

ZTTXC 

and a = (1 — y/c)'^ and b = {1 + ^fcf' when < c < 1. When c > 1, there is an additional Dirac measure 
at a; = of mass 1 — i. Moreover, there are no stray eigenvalues in the sense that the top and bottom 
eigenvalues converge to the edges of the support of F |S] : 



4')-.(l + V^)2 (1.3) 

'Department of Mathematics, University of Michigan, Ann Arbor, MI, 48109, USA, baik@umich.edu 

tDepartmcnt of Mathematics, North Carolina State University, Raleigh, NC, 27695, USA, jack@math.ncsu.edu 
AMS 1991 subject classifications. Primary 15A52, 60F15; secondary 62H99. 

Key words and phrases. Eigenvalues, sample covariance matrices, spiked population models, almost sure limits, non-null case. 



1 



almost surely and [201 



s 



(p) 



min{p,n} 



(l-Vcf (1.4) 



almost surely (s^^^i = . . . Sp^^ — when n < p). One can extract from this some information of the 
population covariance matrix even though the sample covariance matrix is not a good approximate. For 
example, if there are non-zero eigenvalues of the sample covariance matrix well separated from the rest of 
the eigenvalues, one finds, assuming the Gaussian entries, that the samples are not i.i.d.. 

There are indeed many cases in which a few eigenvalues of the sample covariance matrix are separated 
from the rest of the eigenvalues, the latter being packed together as in the support of the Marchenko-Pastur 
function H1.2|l . The examples include speech recognition [HJ^j , mathematical finance ^D, ^M' wireless 
communication j25| . physics of mixture and data analysis and statistical learning 

As a possible explanation for such features, Johnstone |12j proposed the 'spiked population model' where 
all but finitely many eigenvalues of the population covariance matrix are the same, say equal to 1. The 
question is how the eigenvalues of the sample covariance matrix would depend on the non-unit population 
eigenvalues as p, n — > oo. It is known that the Marchenko-Pastur result still holds for the spiked 

model. But and (|1.4|l are not guaranteed and some of the eigenvalues are not necessarily in the support 
of (113. 

For example, consider the case when the population covariance matrix has one non-unit eigenvalue, 
denoted by cri. If cti is close to 1, one would expect that as the dimension p becomes large, the population 
covariance matrix would be close to a large identity matrix, and hence cti would have little effect on the 
eigenvalues of the sample covariance matrix. On the other hand, if ai is much bigger than 1, then even if 
p becomes large, ai might still pull up the eigenvalues of the sample covariance matrix. How big should cti 
be in order to have any effect, how many eigenvalues of the sample covariance matrix would be pulled up 
and exactly where would the puUed-up eigenvalues be ? We will see in the results below that the answers 
are CTi > 1 + ^/c (where ^ ^ c), one eigenvalue at most, and ai + respectively. 

For complex Gauassian samples, the papers |17l l5] study the largest eigenvalue of the sample covariance 
matrix. The authors determine the transition behavior and the limiting distributions are also obtained. The 
purpose of this paper is a complete study of the spiked model for both real and complex samples which are 
not necessarily Gaussian. We obtain almost sure limit results. A general study of 'non-null' covariance 
matrices was done in |3 E] ■ We will show in this paper how to extract the desired results from the work of 
[3] . While this paper was being prepared, the authors learned that Debashis Paul ^] was also studying the 
spiked model independently at the same time, which has some overlap with this work. Paul considers the 
real Gaussian samples for c < 1, and obtains the almost sure limits as in (|1.1U|) and below for large 

sample eigenvalues. Moreover, when all non-unit population eigenvalues are simple, the limiting distribution 
is found to be Gaussian (see Subsection 11.31 below for more detail). On the other hand, our paper (i) is 
concerned with more general samples, not necessarily Gaussian, (ii) includes all choices of c and (iii) studies 
both large and small sample eigenvalues. We remark that a complete study of limiting distributions is still 
an open question. 
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1.1 Model 



Let Tp be a fixed p x p non- negative definite Hermitian matrix. Let Z^j, i, j 
identically distributed complex valued random variables satisfying 



1,2,..., be independent and 



E(Zii)=0, E(|Zii|2) = l, and E{\Ziif)<oo, 



(1-5) 



1/2 

and set Zp = (Zij), 1 < i < p, 1 < j < n. We take the sampled vectors to be the columns of Tp' Zp, where 

1/2 

Tp Zp is an Hermitian square root of Tp. Hence Tp is the population covariance matrix. Of course, not all 
random vectors are realized as such, but this model is still very general. When Zij are i.i.d (real or complex) 
Gaussian, the model becomes the Gaussian samples with population covariance matrix Tp. Outside the 
Gaussian case we see that these vectors cover a broad range of random vectors, completely real or complex, 
with arbitrary population covariance matrix. 
Let 

B„ := -T}/^Zr,ZLT}/^ (1.6) 



rril /T^f T^"! /2 



be the sample covariance matrix, where Zp denotes conjugate transpose. Denote the eigenvalues of Bp by 



„(p). 



for some unitary matrix U b , 
UbBpU^^ = 

V 



\ 



,(p) 



jp) 

Op 



= diag{sf \ 



(1-7) 



For definiteness, we order the eigenvalues as s^^^ > s 

Let ai > ■ ■ ■ > aM > be fixed real numbers for some fixed AI > 0, which is independent of p and n. 
Let fci , . . . , kM be fixed non-negative integers and set r = ki + ■ ■ ■ + k^j , which are also independent of p and 
n. We assume that all the eigenvalues of Tp are 1 except for, say, the first r eigenvalues. This is the 'spiked 
population model' proposed in 1 2 . Let the first r eigenvalues be equal to ai,...,Q!j\/ with multiplicity 
fci, . . . , fcjvf , respectively: for some unitary matrix Ut, 



> ■■■> s 



(p) 



> 0. 



UrTpUj,^ = diag{ ai, . ^. , ai , 0:2, ■ - ^^ , a2^ , 

kl k2 



(1-8) 



We set fco = 0. 



1.2 Results 

Theorem 1.1 (case c < 1). Assume that n — n{p) and p — > 00 such that 

l^c (L9) 

for a constant < c < 1. Let Mq be the number of j 's such that aj > 1 + ^/c, and let M — Mi be the number 
of j 's such that aj < 1 — \fc. Then the following holds. 
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For each 1 <j < Mq 

a," — 1 ' 

almost surely. 



- + l<^<k,. (1.10) 



4^V...+..„+i-(i + ^)' (1-11) 



almost surely. 



s. 



p-r+fciH hfcMi 



(1-V^)2 (1.12) 



almost surely (recall r = ki + ■ ■ ■ + kM )■ 
• For each Mi + 1 < j < M, 

4"i+,,+...+fc^.„,+. - + 1 < z < fc, (1.13) 

almost surely. 

Therefore, when c < 1, in order for a population eigenvalue to contribute a non-trivial effect to the 
eigenvalues of the sample covariance matrix, it should sufficiently big (larger than 1 + -/c) or sufSciently 

small (less than 1 — \/c). As an example, when r = 1, by denoting the only non-unit eigenvalue by <t\, the 

iv) 

largest sample eigenvalue s\[ satisfies 



{l + ^cf, ni<l + ^c ^^^^^ 



almost surely. When r = 2, by denoting the two non-unit eigenvalues by CTi, (72, the largest sample eigenvalue 

(1.15) 



satisfies 



(p) ^^^^^^^^^ 

(1 -f Vc)2, max{f7i, (72} < 1 + 

c max{o"i .(72 } 



Sp) 



r T I c maxi o"! .(72 \ r ^ ^ \ r~ 

^ maxjo-i , g2 } + }1\ , max{cri,(72} > 1 + 

almost surely. 

The results H1.10|l and (jl.ll|l are also independently obtained in under the assumption that the 
samples are Gaussian. 

Theorem 1.2 (case c > 1). Assume that n = n{p) and p — > oo such that 

^^c (1.16) 
n 

for a constant c > 1. Let Mq he the number of j 's such that aj > 1 + \fc. Then the following holds. 
• For each I < j < Mq, 

a-i — 1 

almost surely. 



-^aj + ^^, l<i<kj. (1.17) 
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almost surely 



^,V+...+.Mo+i-(l + VS)' (1-18) 



s(f)^(l-VSf (1.19) 

almost surely. 
• For all p, 

s!fii = --- = s(^^=0. (1.20) 
Thus, unlike the case of c < 1, smaU eigenvalues of Tp do not affect the eigenvalues of Bp when c > 1. 
Theorem 1.3 (case c — 1). Assume that n — n{p) and p ^ oo such that 

^^l. (1.21) 

n 

Let Mq be the number of j 's such that aj > 2. Then the following holds. 



For each 1 < j < Mq, 
almost surely. 



+ ^<^<kJ. (1.22) 



almost surely. 



4':V...+.„„+i-4 (1.23) 



s^P} , , ^ (1.24) 

mini n,p| v ^ 

almost surely. 
1.3 Discussion 

As mentioned earlier, the limiting density of the eigenvalues of spiked population models is given by the 
Marchenko-Pastur theorem p.2|l as in the identity population matrix case, and for the top eigenvalue s^^' 
in the complex Gaussian case, the results Theorem 1.1 and Theorem 1.3 were first obtained in |17l 15]. The 
paper (see section 6) contains an interesting heuristic argument for the critical value 1 + -v/c and the value 

(1.25) 



aj - 1 



for 1 < j < Mq above: they come from a competition between a 1-dimensional last passage time and a 
2-dimensional last passage time. It would be interesting to have such a heuristic reasoning for the general 
case. 
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When Tp is the identity matrix (the 'null case'), under the Gaussian assumption, the limiting distribution 
for the largest eigenvalue is obtained for the complex case in [7| and for the real case in ^21 ■ [23 shows 
the Gaussian assumption is not necessary when c = 1. The limiting distributions are the Tracy- Widom 
distributions [261 12 7| in the random matrix theory in mathematical physics. For the spiked model with 
complex Gaussian samples when c < 1, the limiting distributions of the largest eigenvalue are obtained 
in ^1 The paper [S] determines the limiting distribution of s'f ^ for complete choices of the largest 
population eigenvalue ai and its multiplicity fci: the distribution is (i) the Tracy- Widom distribution when 
ai < 1 + ^/c, (ii) certain generalizations of the Tracy- Widom distribution (see also ^) when ai = 1 + y/c, 
and (iii) the Gaussian distribution (fci = 1) and its generalization (fci > 2, the Gaussian unitary ensemble) 
when ai > 1 + ^fc. For real Gaussian samples JSI showed that when c < 1, Mq > 1 and ki — ■ ■ ■ ~ — 1, 
the limiting distribution of , 1 < j < Mq, is Gaussian. It is an interesting open question to determine 
the limiting distribution for the general case of real samples. See section 1.3 of jHI for a conjecture for the 
scaling. 




Figure 1: Gaussian samples when p — 1000, n = 2000 

We include several plots for the case when c = 0.5 and there are three non-unit population eigenval- 
ues given by 0.1, 3 and 4 (of multiplicity 1 each). In this case, the critical values of the eigenvalues are 
1 -|- y/c ~ 1.70711 and 1 — ^/c ~ 0.29289. Hence theoretically we expect that three sample covariance 
eigenvalues of values aj + -^--^i — 0.04444,3.75 and 4.66667 are away from the interval [(1 — a/c)^, (1 + 
y/c)'^] ~ [0.08578,2.91422]. The histogram and the scatterplot of Figure ^ is from Gaussian samples when 
p — 1000; n — 2000. The smooth curve is the theoretical limiting density and the theoretical locations of the 
three separated eigenvalues are plotted with -I- signs below the horizontal axis. The smallest and largest two 
sample eigenvalues are plotted with -I- signs about the horizontal axis. Figure |21 is from Gaussian samples 
when p = 100, n = 200 while Figures 01 and 01 from samples of Bernoulli variables taking values —1 or 1 when 
p — 1000, n — 2000, and p = 100, n = 200 respectively. The observed values of the four separated eigenvalues 
in each case are as follows: 
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Figure 2: Gaussian samples when p ~ 100, n — 200 





smallest eigenvalue 


2nd largest eigenvalue 


largest eigenvalue 


theoretical 


0.04444 


3.75 


4.66667 


Gaussian p = 1000 


0.04369 


3.78400 


4.59127 


Gaussian p = 100 


0.03979 


3.55388 


4.66192 


Bernoulli p = 1000 


0.04555 


3.75706 


4.66594 


Bernoulli p = 100 


0.05015 


3.62337 


4.70786 



Figure[3and FigureElare the cases when c — 2, p — 2000, n — 1000 with Gaussian and Bernoulli samples, 
respectively. Again three non-unit population eigenvalues are chosen 0.1, 3 and 4. The critical value of the 
eigenvalues is 1 + ^/c ~ 2.41421 and the theory predicts that the two largest sample eigenvalues given by 
ttj + - 6 and 6.66667 are separated from the interval [(1 - Vc)^ (1 + Vc)^] - [0.17157, 3.41209]. Only 
non-zero eigenvalues are plotted in Figure El and Figure El The observed values of the separated eigenvalues 
in each case are as follows: 





2nd largest eigenvalue 


largest eigenvalue 


theoretical 


6 


6.66667 


Gaussian p — 2000 


5.8523 


6.4013 


BernouUi p = 2000 


6.01065 


6.725 



The paper is organized as follows. In section|21 we summarize the work of Z. D. Bai and J. W. Silverstein 
on which we heavily rely to prove our results. It turns out that the determination of the support of a Stieltjes 
transform plays the crucial role. This is obtained in sectional The proofs of the main theorems are given in 
sectional 
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Figure 3: Bernoulli samples taking values —1 or 1 when p = 1000, n — 2000 
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Our analysis replies heavily on the work [3 El of Bai and Silverstein. In this section, we summarize the 
necessary results from 

Notational Remark. We denote by p the population size and by n the sample size. The notations n and N 
are used in for p and n, respectively. 

For a distribution function G{X), its Stieltjes transform niQ{z) is defined by 



2 Results of Z. D. Bai and J. W. Silverstein 





(2.1) 



Also recall the inversion formula 




(2.2) 



for continuity points a, h of G. 
Assume the following: 



(a) Zij are i.i.d. random variables in C with E(Zii) = 0, EjZup = 1 and E|Zii|'* < oo. 



(b) n — n{p) with Cp :— ^ 



c > as p ^ oo. 



8 



Figure 4: Bernoulli samples taking values —1 or 1 when p = 100, n — 200 



(c) For each p, UtTpU^^ — diag{a'f\ . . . ,a^^) for some unitary matrix Ut such that Hp —^ iJoo hi 
distribution for some distribution function iJoo where Hp is the empirical distribution function of the 
eigenvalues of Tp defined by 

dHpi\)^-J2Ki^A^)- (2-3) 



} is bounded in p. 

(e) Set Zp = (Zy ), 1 < i < p, 1 < j < n and Bp = ^Tp^^ ZpZ;Tp^\ 

(f) Set 

1 ft 

Zp{m) = + cp / dHp{t). (2.4) 

/,From |22| and |21j. it is known that there is a unique inverse function mp{z) such that mp{z) e C+ 
for z € C+ It is also known [221 12] that mp{z) is the Stieltjes transform of a distribution, which will 
be denoted by Fp-. 

mp{z) = / dFp{X), z e C+. (2.5) 

A — z 



— oo 



Suppose that the interval [a, b] with a > lies in an open interval outside the support of Fp for all 
large p. 

Now we will state the main result of !3" which we need for our analysis. It is easy to check that Fp 
converges to some distribution function F^^- Then |21| i^oo(-^) is the almost sure limit of the empirical 
spectral distribution of Hp ^ZpTpZp and 

F(A) :=^(^^oo(A)-(l-c)l[o,oo)) (2.6) 
is the almost sure limit of the empirical spectral distribution of Bp = ^Tp^"^ ZpZpTp^"^ . 
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Figure 5: Gaussian samples when p — 2000, n — 1000 



Remark. The function Fp is not the empirical distribution of Bp. The distribution function Fp is defined 
only through (|2.5I) . 

Moreover 0, the Stieltjes transform of Foo, 

m^{z)= f -^dFoo(A), zeC+:^{zeC:Im{z)>0}, (2.7) 
is invertible, with the inverse given by 

Zoo(m) = -— + c I -^—dH^it). (2.8) 
m J 1 + rm 

On the other hand, given i?oo, -Foo is determined from H2.8|l and (|2.7(l . Note that moo(z) is well-defined not 
only on C+ but also up to the real line outside supp(i^oo) and its inverse exists on TOoo(C+ U supp(_F'oo)'^). 
Remark. If [a, 6] satisfies condition (f) above, it is easy to check that [a, 5] C supp(i^oo)'^- 

Given an interval [a, 5] satisfying condition (f) above and mao{b) < 0, it is shown in |3] that there is an 
integer ip > satisfying the conditions 

^^^'^ > ^^'li < (2-9) 

moo (6) moo (a) 

for large p. (Here cTg^' := oo.) 

Proposition 2.1 (Theorem 1.2 A ssume (a)-(f) above. 

(i) If c{l — -ffoo(O)) > 1, then xq, the smallest value in the support of Foe, is positive, and sl^^ —> xq with 
probability 1 . The value Xq is the maximum of the function Zoo (™) for m G K+ . 

(ii) If c{l — i?oo(0)) < 1 or c(l — i?oo(0)) > 1 but [a,b] is not contained in [0,a;o], then moo{b) < and 

F{s^^J > b and s[^''^^ < a for all large p) = 1 (2.10) 

with ip defined in (|2.9|l . (Here s'^^ :— oo.) 
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Figure 6: Bernoulli samples taking values —1 or 1 when p = 2000, n — 1000 



3 Determination of supp(Fp) 

The key part in applying Proposition 12.11 turns out to be determining the support of Fp. This can be 
extracted from the following result due to Silverstein and Choi. 

Lemma 3.1 ([23 ; see also Lemma 1.3 [3j). If x ^ supp(Fp), then m := mp{x) satisfies 
(i) m e M \ {0} 
(") -7^^ supp(i/p) 
(Hi) Zp(m) > 0. 

Conversely, if m satisfies (i)-(iii), then x — Zp(m) ^ supp(i^p). 

Remark. Lemma 1.3 of 3[ is stated for Hoo- But the proof of Lemma 1.3 in 23 applies also to the finite p 
case of Hp without any change. Indeed, the proposition applies to any distribution defined by its Stieltjes 
transform as in (|2.5|l . 

Remark. It is also shown in ^23^ that Fp has continuous density on R_(_. 
When Tp is as in (fO]l . 




(3.1) 



and 




(3.2) 



where we recall that r ^ ki + ■ ■ ■ + fcjvf- We first determine the set of real m such that z' (m) > 0. 
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Now 



z'{m) = — 



M 



im? (1 + m)'^ 



^ (1 + ajm)2 (1 + my 



(3.3) 



where 



and 



M 



/(to) := ((1 + to)2 - CpTO^) [|(1 + a£TO)2 



£=1 



5r(TO) 



M 



(1 + q;jto)2 (1 + my 



M 



TO^(1 + mf ]^(1 + a^TO)^ 



(3.4) 



(3.5) 



£=1 



We need the foUowing basic lemmas of complex variables to determine the solution of z' (m) — 0. 



Lemma 3.2. Let h{z) he an analytic function in a closed disk D{zq, r) of radius r > centered at zq. Then 
there is eq > such that for < e < eo, the equation 



z — zq ~ eh{z) 
has a unique solution in D{zo,r), which satisfies 

z = zq + eh{zo) + O(e^). 
Furthermore, if Zq is real and h(z) is real for real z, the solution (|3.7|) is real. 



(3.6) 



(3.7) 



Proof. As h is continuous, there is a constant C > such that \h{z)\ < C for \z — zo\ < r. When |e| < ^, 

for I2; — zo| — T, 

|z-zo| =r > |e|C> |e/i(z)|. (3.8) 

Hence from Rouche's theorem, the number of zeros of z — zq — eh{z) inside D{zq, r) is equal to the number 
of zeros of z — zq inside -D(zo,r), which is one. The zero z^ satisfies Ze — zq = eh{z^) — 0(e). Thus 



z^- Zq - eh{zo) = e{h{z^) - /i(zo)) = O(e^). 



(3.9) 



If h(z) is real for real z, then by taking complex conjugate of (|3.6|l . we find that z^ is also a solution. Since 
there is only one solution, we find that z^^ is real. □ 



Lemma 3.3. Let h{z) he an analytic function in a closed disk D{zo,r) of radius r > centered at zq such 
that h(zo) 7^ 0. Then there are < ro < r and eo > such that for < e < eo, the equation 



(z - zo)^ = e/i(z) 

have precisely two distinct solutions in Z3(zo,ro), which satisfy 



z = zo ± y/e^/h{zo) + 0{e) 



(3.10) 



(3.11) 



where y^h{zo) is an arbitrary branch. Furthermore, suppose that zq is real and h{z) is real for real z. Then if 
h{zQ) > 0, both solutions (|3.11() are real. On the other hand, if h{zQ) < 0, both solutions (|3.11l) are non-real. 
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Proof. The proof of H3.11|l follows from Lemma \'6.2\ bv taking the square root of (|3.1U|) . When zq is real 
and h{z) is real for real z, the complex conjugate of a solution of (|3.10() is also a solution. Thus the two 
solutions (|3.11|l of H3.10|) are either complex conjugates of each other or both real since there are precisely 
two distinct solutions. Hence the Lemma follows. □ 

For the remainder of this section, we assume that c 1 and none of aj's are equal to 1 ± y/c. We further 
assume that p and n are sufficiently large so that Cp ^ 1 and none of aj 's are equal to 1 ± y/c^- Then the 
numerator of (|3.3I) is a polynomial of degree exactly 2M + 2, and we now determine all the solutions of 
4(m) = 0. 

For / defined in 1)3.4(1. the equation /(m) = has distinct solutions 

-1 -1 



1 + ^ 



m+, 



m 



1 - 



of multiplicity 1 and 



(3.12) 



(3.13) 



of multiplicity 2. The roots of Zp{m) are expected to be perturbations of the roots of f{m), which we will 
find. First consider to_|_. Dividing the equation f{m) + —g{m) = by ^^"^ , we obtain the equation 



m — 171+ H — 



m^(l + m)^ 



n (1 — Cp){m — m_) 



E 



^ (1 + ajni)^ (1 + my 



= 0. 



Lemma 13.21 implies that there is a solution of Zp{ni) = of the form 

m = 771+ + 01 — ), 
which is real. Similarly, there is a real solution of Zp{m) = of the form 

777 = 777_ + O 

Now 



(3.14) 



(3.15) 



(3.16) 



consider the root 777, = of f{m) — 0. Dividing / + ;^.9 = by ; obtain the equation 



1 



1 



0,(777) 



where 



Note that 



0,(777) 



— (1 + a," 777) ^777^(1 + 777)^ 
■ (1 — Cp){rn — m^){m — mJ) 



M 



-kgoj 



^ (1 + agmY (1 + mY 



GA--) = 



kj{aj — 1)^ 



(3.17) 



(3.18) 



(3.19) 



^ "i "j(l-Cp)(-s7-"t+)(-^-r77_) 

is not zero and also Gj{m) is real for real 777. Thus Lemma 13.31 implies that there are precisely two solutions 
of Zp{m) = of the form 



,Af, 



(3.20) 
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where the pah' for each j are either both real or both non-real dependmg on the sign of G'j(— 
Now when Cp < 1, the condition ^) > is equivalent to 

-1-1 , ^ 
> or < m_, (3-21) 

which is the same as 

aj > 1 + ^Jc^ or Qfj < 1 — -^c^. (3.22) 

On the other hand, when Cp > 1, we note that m+ < < m_. The condition Gj (—77-) > is now equivalent 
to 

m+< — <TO_, (3.23) 

which is the same as (since ctj > 0) 

aj >l + (3.24) 

We summarize the above calculations. 
Lemma 3.4. The solutions of z' {m) — are 



1 0(i)=:„^(;), ^ = __!_ + o( i ) =:^(_"). (3.25) 



1 + V"-/ 1 ~ 



1 _^ 1 



^G,(-^)+o(i) j = l,...,M, (3.26) 



0/ multiplicity 1 . Furthermore, the following holds. 

• When Cp < 1, mf!'^ < m'^'^ < 0, and rn'^J. are real if and only if a-,- > 1 + ^Jc^ or aj < 1 — ^Jc^- If 

(n) (n) 

1 — y/c^ < aj < 1 + y/c^, TTij f_ and nij _ are complex conjugates of each other. 

• When Cp > 1, m^"'^ < < m''"\ and m^"2 'J'^e real if and only if aj > 1 + y/c^. If aj < 1 + ,/c^, 'm^j% 

(n) 

and m)j _ are complex conjugates of each other. 
We now consider the cases when c < 1 and when c > 1 separately. 

3.1 When c < 1 

Let the indices < A/q, Mi < M be defined as in Theorem II . II frecall that we assume that none of the a^-'s 
are equal to 1 ± -^c) , so that 

ai > • • • > aMo > 1 + \/c > aMa+i > ■ ■ ■ > ctM-Ah > 1 - Vc > aM-Mi+i > ■ ■ ■ > au- (3.27) 

We now find the intervals in which Zp(m) > 0. 

The denominator of H3.3|l is non-negative. ^From Lemma [3. 41 the numerator of 1)3. 3|l is factored as 

M 

const ■ (m — rn'^^){m — m^'') JJ(™ — m^"2)(m — ro'j^}^) (3.28) 
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The constant prefactor is, from (|3.4|) and 1)3. 5|) . 



(l-Cp)n"' + - , (3.29) 



which is positive when n is large enough. On the other hand, among the terms in the product of H3.28|l . 

(71) 

rrij J_ corresponding to the indices Mq + 1 < j < Mi are complex conjugates of each other. Thus 

Mi + l 

Yl (m-mj"2)(m-TOj"|) >0. (3.30) 

j=Mo + l 

Hence using the fact that 

> > ml _ >•••> m)^jl^ ^ > m]^/^ _ > ' (3.31) 

and 

(n) (n) (n) (n) (n) /„ „r<\ 

> > > • • ■ > > ml/-, (3.32) 

we find that the numerator of (|3.3|l is positve in the intervals 

(-oo,mj^"|_) U (m^"|_^,m^"Li,_) U • • • U (m^"^^2,+ ' "^m!+i,-) ^ (mj^^^^^^, toL"^) (3.33) 

union 

(™("\mg^J U (™2,+,m(';)_, J U . . . U (™("|,m("!) U (ml"|,oo). (3.34) 

The singular points of (|3.3|l are not contained in any of the above intervals except for the singular point 
m ~ 0. Hence the set of m such that z'p{m) > is equal to l|3.33|l union 

(m^+Km^M^^,.) U (mit+,4';i-i,-) U • • • U (m^^lm^-l) U (m^-lo) U (0,oo). (3.35) 

Now Lemma l3. II determines supp(i^p). 

Proposition 3.5. Suppose that c < 1 and none of aj is equal to 1 ± ^/c. With the indices Mq and Mi 
defined in Theorem M.lX for n sufficiently large, 



snMFpf H'^. 0) U (0, zt,l) U {zZI^z'mU-) U • • • U (4;Vi,+, ^-0 

U (4"\ zS,_) U (zS.H., zS^i, J U . . . U {z^^lzt:!) U (z("j, ^) 



(3.36) 



whe 



z^^^ ^{l±^f + o(^ (3.37) 

=a, + ^^±4^ + of-Y j = l,...,Mo, j=Mi + l,...,M, (3.38) 
for some constant Aj > 0. The intervals in H3.36|l are disjoint. 



and 
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Proof. We will first see that the intervals H3.33|l union (|3.35|) satisfy the conditions (i)-(iii) of Lemma f3. II 
The condition (iii) is clearly satisfied. Also is not contained in (|3.33() and H3.35() . and so condition (i) is 
fuUfilled. Finally, as supp(_f/p) = {ai, . . . , a^/, 1} and 

(") ^ 1 ^ (") (") ^ ^ ^ (") /o on^ 

m_'<— l<ml , m) _ < < m) !, (3.39) 

the condition (ii) is satisfied for m in H3.33|l union H3.35|l . 

We now need to find the image of the above intervals under Zp. Clearly, Zp{—oo) = 0, Zp(0— ) = +oo, 
Zp(0+) = — oo and Zp(+oo) = 0. A direct computation yields 

Zp(mi")) = (l±VZ^)2 + o(^l^. (3.40) 

and 

.,,42) = „, + JiS_±A + o(i) (3.41) 

where 



1 I i2„2 



^j = -i ci^ (i - (;;;^) + fc^j' ^= (3.42) 



Note that Aj > for I < j < Mq and Mi + I < j < M since aj > 1 + ^Jc^ or < 1 — ^Jc^. Also it is 
straightforward to check from the graph of the function 

'''''' (3.43) 



that 

< QfM H < ■ ■ • < aui+\ H < (1 - Jcp) 

< (1 + a/S^) < "Afo + ^ T < • • • < "1 + 

aMo - 1 ai - 1 

This implies the Proposition. □ 



3.2 When c> 1 

This case is similar to the previous case when c < 1. We indicate only the difference. 

We again assume that -p and n are large enough so that the set of j's satisfying > 1 + ^Jc^ is the same 
as the set of j's satisfying > 1 + ^Jc. Let the index < Mq < M be defined, as in Theorem ll.il We 
further assume that none of is equal to 1 + \fc so that 

OLMa > 1 + Vc > aAfo + 1- (3.45) 

The denominator of 13.3|l is non-negative and as before, the numerator of H3.3|l is equal to (|3.28|) . But 
this time, the constant prefactor H3.29|l is negative when n is large enough. Also as in H3.30|l . 

M 

]J (m-m^."2)(m-mj."]) >0. (3.46) 

j=Mo + l 
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Now using the fact that 

m_ > ra\'^> m\ '__>■■■ > mi/^ ^ > m\,/^ _ > to^ , (3.47) 
we find that the numerator of (|3.3|l is positive in the intervals 

{n^Kn^^J U (m(^),^,m(^)_,,J U • • • U {mi%m[':l) U {m[%m^^\ (3.48) 
Hence taking into accounts of the singular point m = of Zp{m), the intervals where Zp{m) > is 

{m^:\m^;lj U K'}^,^,mi'J^_i^J U • • • U (™^:tm("!) U {m[%0) U (0,m("'). (3.49) 
The proof of the following proposition is parallel to Proposition 13. 51 

Proposition 3.6. Suppose that c > 1 and none of aj is equal to 1 + ^/c. With the index Mq defined in 
Theorem \l/A for n sufficiently large, 

supp(Fp)^ ^{-^, zi")) U (zl"\ zg__) U , z2_i__) U . . . U ^2) U (zS, oo) (3.50) 

where 



(3.51) 



.(:2^.,^--^±^ + 0(ij, ,.l,...,Mo, (3.52) 
for some constant Aj > 0. The intervals in H3.5()|l are disjoint. 



4 Proof of Theorems [TTl O] and Ol 

When Tp is (ll.8|l . as i/p is equal to H3.1|l . 

dH^{x)=Si{x). (4.1) 

Hence 

^oo(™) = --+c di7ooW = -- + -^. (4.2) 

m J 1 + TO 1 + m 

It is well-known that ((22), see also Theorem 3.4 of 1 ) in this case, 



dF^iX) = { 57aV((iT7^F^A)(A^(W^1[(i-v^)Mi+v^)^](A), c> 1 ^^^^ 

5i^v/((l + - A)(A - (1 - Vcr)hi-V-cV,ii+V-cf]W + (1 - c)'^o, 0<c<l. 

4.1 When c < 1 

We first assume that none of aj is equal to 1 ± y/c so that Proposition l3.5l is applicable. The case when some 
of aj are equal to 1 ± ^/c will be discussed at the end of this subsection. 

When Tp is (|1.8|l . all the conditions (a)-(e) of Proposition 12 . II are satisfied or are defined accordingly. 
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Now suppose [a, 6] is an interval satisfying condition (f). Since 

4") ^(1 + ^)2, (4.4) 

and for any i, 



ztl z^^a. + ^, (4.5) 



we see that 



i,5J C(-oo,0) U 0,q;a/ H U [aM H -,aM-i H r 

f«M.+i + ^^^,(l-x^)^) (4.6) 



u • • • u 

U (1 + Vc)^ aA/o + ^ U • • • U Us + ^, ai + — U Ui + 



, oo 



ctMo — ^ J V " a2 — 1 ai — 1 J \ ai — 1 
On the other hand, 

supp(Fo,)^ = (-00, 0) U (0, (1 - V^)2) U ((1 + V^)2, ex.) . (4.7) 

Hence [a,b] C supp(i^oo)'^- Also from definition (|2.7|) . it is easy to see that m^(z) > for z £ supp(Foo)'^. 
The first consequence of (ii) of Proposition 12. II fnote that i?oo(0) = 0) is that modb) < 0. Thus moo(a) < 
n^ooib) < 0. Therefore, the condition (|2.9|l is equivalent to the condition 

[a,b] C [zeo(-l/aJ|i),;.cc^(-l/alf)]. (4.8) 

We will consider four different choices of [a,b]. First fix 1 < j < Mq. Take 

[a, 6] = + ^ + e, + ^ - e (4.9) 

for an arbitrary fixed e > 0. (Here ao +00.) From 14.5|1 . we see that 

[a,6]c(z]"_^,z]!\^J (4.10) 
for all large p, and hence condition (f) is satisfied using Proposition l3.5l Set 

ip:=ki-\ hfcj^i. (4.11) 

(When j = 1, ip := 0.) For Tp given by ffT^ . 

-Si=«.- (4.12) 

But 

^oo(-lM) = + — f- (4.13) 

and hence the condition 1)4. 8|) is satisfied. Therefore ip is defined to satisfy the condition 1)2. 9|l . Proposition 
5] now implies that 

4f+...+fc,_i > "j-i + ^^^^frh " ' ""''"^ 4?+...+fc,_i+i < + + £ for all large p\ = 1. (4.14) 
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This yields that, 1 < j < ^^o - 1, 

+ - ' < 4'V...+fe.-i+fc. < • • • < < + + e for aU large pj=l, (4.15) 

which implies p.lO|l for 1 < j < A/q — 1, and 



< «Mo + + e for all large p\=l. (4.16) 



For the second choice of [a,b], set 



[a, b] = [(1 + V~cr + auo + - e] (4.17) 

aMo ~ 1 

for an arbitrary fixed e > 0. Noting that 

zi") ^ (1 + Vcf (4.18) 
and setting ip :— ki + ■ ■ ■ + fc^/o, a calculation similar to the above yields that 



/ "Mo ^ ^ - e and 4^V-+feMo+i < (1 + V^)' + e for all large p] =1. (4.19) 

Thus, together with H4.16|l . we obtain (|1.1U|) for j = Mq. Also as (|4.2|l and discussions around H2.6|) implies 
that the support of the limiting spectral distribution of Bp is [(1 — \/c)'^, (1 + \/c)'^], we obtain p.ll|l . 
As the third and forth choices of [a,b], we set 



[a, t/j = |UA/i + l -t - 

c 

and 



a, 5] = [aM,+i + ' + e, (1 - yfcf - e] (4.20) 



[a, b] = a,+i + + 6, + 1- - 6 (4.21) 



for some Mi + 1 < j < M (q!m+i 0), respectively. Arguments as above imply the remaining part of 
Theorem O 

We now consider the case when an aj is equal to 1 ± ^/c. We first observe certain monotonicity of the 
eigenvalues s^''"' on ajs. Note that the matrix B_p := ^ZpTpZp has the same set of eigenvalues as Bp except 
for \p — n\ zero eigenvalues. Consider a set of parameters Pj, 1 < j < ill, such that aj > Pj. Let Tp be the 
matrix Tp with a^'s replaced by /?j's, and set Bp = j^Tp^^ ZpZpTp^^ and B_p — j^ZpTpZp. Then clearly, B_p 
and -Bp are Hermitian, and B_p > B_p. Hence from the min-max principle (see e.g. j9 ), we find that 

> (4.22) 

(v) 

for all non-zero eigenvalues, where denotes the eigenvalues of Bp. 
Suppose that 

ai > ■ ■■ > aMo > 1 + Vc = QfMo+i > ■ • ■ > aM-Mt > 1 - \/c > aM-Ah+i > ■ ■■ > aM- (4.23) 

Replacing in H4.23|) aMo+i by (1 ± e)ctA/o+i = (1 =1= e)(l + y/c) for sufficiently small e > 0, the above 
monotonicity argument implies the following: 
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(i) For each 1 < j < Mq, 



< liminf , < limsups^7+...+. < a, + 1 < z < fc,. (4.24) 



^ Q^j — I ~ h/Cj-l+l — """-i" "Kli hKj-l+l — "V ■ Q,^. _ 1 ' 

almost surely. 



(ii) 



(1 + < liminf 4^V...+.M„+i < limsup4^)_,...^,^^^^, < (1 + e)a,,,^, + ^f^^^^l^lT- I ^''''^ 
almost surely. 



(iii) 



(1 - Vcf < liminf 4^^,^,^+...+,^^^ < limsup^'^,^,^^...^,^^^ < (1 - V~cf (4.26) 

almost surely. 



(iv) For each Mi + 1 < j < M, 



— — < limmi s'' I , , ,, . 



Cn") CO;,- can 

< limsups;':^,+,^+...+,.^_^+^ ^ a, + < a, + — ^ 1 < i < k, 



(4.27) 



almost surely. 

Since 

lim(l + .)aMo+i + + = (1 + VS)2 (4.28) 

eio (1 + e)aMo+i - 1 

and the above result is true for arbitrary sufficiently small e > 0, Theorem 11.11 follows for the case when the 

parameters are given by (|4.23() . For the case when om-Mi = 1 ^ Vc, the argument is almost identical, and 

we skip the details. 

4.2 When c> 1 

^From H4.3() . when c > 1, the smallest value in the support of Faa is 

^0 = (1 - V^? > 0. (4.29) 

Hence Proposition 12. II (i) implies that 



(p) 

Since when p > n, at least p — n eigevalues s are equal to 0, we conclude that 



siP^^{l-Vc?- (4.30) 
Y are equal to 0, we conclude that 

(fj, = ... = s(rf=0. (4.31) 



Therefore, H1.19|l and H1.20|l are obtained. 
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The proof of (|1.17|l and (|1.18|l is similar to the case when c < 1 by using Proposition 13 . 61 and noting that 
an interval [a,b] satisfying condition (f) of Proposition [TTI is contained in 



cai 

U I Q!i H -,oo 

ai — 1 



V "Afo -1/ V "2-1 "1-1/ V 

(4.32) 

which is a subset of 

supp(f^oo)^ = (-00, (1 - V^f) U ((1 + (4.33) 

4.3 When c = 1 

Since the limiting distribution H1.2|l for c = 1 has a continuous density on the interval (0, 4), it is easy to see 

We first observe a monotonicity of s^^"* in n. Let Zp = [Zij), l<i<p,l<j<n and let Bp := 
iTp^^ZpZ'pTp^^. When n > n, it is clear that 

fiBp > nBp. (4.34) 
Therefore, if the ordered eigenvalues of Bp are denoted by the min-max principle implies that 

nsf > nsf (4.35) 

for all I < j < p. 

Take n — [j^] for e > where [x] denotes the largest integer < x. Then for sufficiently small e > 0, 

ai > ■ ■ ■ > ttMo > 1 + \/l + e > oiMo+i > • • • > OiM- (4.36) 
By applying Theorem 11.21 and using H4.35(l . we obtain the following: 
• For each 1 < j < Mo, 

i^inf 4';V...+fe,_.+, > ("^" + ^^pr^) ' ^<'<'^^- (4-37) 

almost surely. 

liminf si'^V-.+.M^+i > Y^fi + (4-38) 

almost surely. 

On the other hand, take n = [jzr^] for e > 0. We first assume 2 > aMo+i- Then as aM > 0, for 
sufficiently small e > 0, 



ai > • • • > aMo > 1 + \/l - e > ctAio+i > ■ ■ ■ > cum > 1 - \/l - e, (4.39) 
and hence Mi = M. By applying Theorem ll.il and using (|4.35(l . we obtain the following: 
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• For each 1 < j < Mq, 

linisup4';V...+.,-.+. ^ T^(«^ + ^^-ZT^)' 1 ^ * ^ (4.40) 

almost surely. 

almost surely. 
If aMo+i — 2, then for sufficiently small e > 0, 

ai > • • • > QfA/o > ttA/o+i > 1 + \/l - e > aMo+2 > • • • > ckAf > 1 - Vl - e- (4.42) 
Hence Theorem 11.11 implies (|4.4UII but (|4.41ll becomes 

almost surely. 

Therefore KT!^ and (|On|l yield ifL^ . and |(13Hl, and l(^J^ yield (fT^ . 
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